Specification-Based Testing of Interactive Systems

Achieving correct user interface software is diicult because such software is complex, highly interactive, modeless, concurrent, graphical, and has user-based real-time requirements. In this paper I propose developing a framework for applying formal methods to testing of user interface software. I survey relevant work in the areas of formal development of user interface software and speciication-based testing. I then outline a case study based research plan to extend an existing speciication-based testing framework to incorporate multiparadigm user interface speciications. A user interface (UI) is some boundary between a computer system, comprised of hardware and software , and a human user. UI software is a signiicant component of contemporary systems and graphical user interfaces (GUIs) are now almost universal. Conndence in the correctness of UI software is usually achieved by prototyping or by testing. Software correctness is commonly described as validation (\Are we building the right software?") and veriica-tion (\Are we building the software right?"). Proto-typing can be used for validation and to verify that the UI meets usability requirements. Testing can also be used for validation (acceptance test) and to evaluate usability. Traditional software testing (unit, integration and system test) is the major method for verifying the correctness of UI software. Testing UI software is diicult however, because such software Ian MacColl is supported by an Australian Postrgradu-ate Award and a Telstra Research Laboratories Postgraduate Fellowship. is complex, highly interactive, modeless, concurrent, graphical and has user-based real-time requirements. Industrial strength tools have been developed to automate UI testing. These tools provide facilities such as capture/playback for test development, bitmap-and widget-based test evaluation, and tex-tual and point-and-click scripting. Little support is available, however, for generating test inputs or expected outputs (oracles). Formal methods are also used to develop correct software, including UI software. Fully formal development is advocated for projects such as life-critical software, where the cost of failure is greater than the additional cost of formal development (although industrial experience indicates that additional costs are ooset by other savings). Formal methods have been combined with software testing to provide speciication-based testing techniques that use a (formal) speciication to support testing. Little use has been made of these techniques for UI testing to date.